{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "635d8ebb",
   "metadata": {},
   "source": [
    "# FastAPI Serving\n",
    "\n",
    "- Author: [Donghak Lee](https://github.com/stsr1284)\n",
    "- Peer Review: \n",
    "- Proofread : [frimer](https://github.com/brian604)\n",
    "- This is a part of [LangChain Open Tutorial](https://github.com/LangChain-OpenTutorial/LangChain-OpenTutorial)\n",
    "\n",
    "[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/LangChain-OpenTutorial/LangChain-OpenTutorial/blob/main/19-Cookbook/08-Serving/01-FastAPI-Serving.ipynb)[![Open in GitHub](https://img.shields.io/badge/Open%20in%20GitHub-181717?style=flat-square&logo=github&logoColor=white)](https://github.com/LangChain-OpenTutorial/LangChain-OpenTutorial/blob/main/19-Cookbook/08-Serving/01-FastAPI-Serving.ipynb)\n",
    "## Overview\n",
    "\n",
    "This tutorial is about FastAPI Serving.\n",
    "FastAPI is one of the python web frameworks that supports asynchronous programming and is very fast.\n",
    "\n",
    "In this tutorial, we will implement the following FastAPI examples.\n",
    "- Implement different types of parameters\n",
    "- Declare an input/output data model\n",
    "- Serve a langchain with FastAPI\n",
    "\n",
    "### Table of Contents\n",
    "\n",
    "- [Overview](#overview)\n",
    "- [Environment Setup](#environment-setup)\n",
    "- [What is FastAPI](#what-is-fastapi)\n",
    "- [FastAPI Fast Tutorial](#fastapi-fast-tutorial)\n",
    "- [FastAPI Serving of LangChain](#fastapi-serving-of-langchain)\n",
    "\n",
    "### References\n",
    "\n",
    "- [FastAPI](https://fastapi.tiangolo.com/)\n",
    "- [langchain_reference](https://python.langchain.com/api_reference/index.html#)\n",
    "----"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6c7aba4",
   "metadata": {},
   "source": [
    "## Environment Setup\n",
    "\n",
    "Set up the environment. You may refer to [Environment Setup](https://wikidocs.net/257836) for more details.\n",
    "\n",
    "**[Note]**\n",
    "- ```langchain-opentutorial``` is a package that provides a set of easy-to-use environment setup, useful functions and utilities for tutorials. \n",
    "- You can checkout the [```langchain-opentutorial```](https://github.com/LangChain-OpenTutorial/langchain-opentutorial-pypi) for more details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "21943adb",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture --no-stderr\n",
    "%pip install langchain-opentutorial"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f25ec196",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m24.3.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m25.0.1\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "# Install required packages\n",
    "from langchain_opentutorial import package\n",
    "\n",
    "package.install(\n",
    "    [\n",
    "        \"uvicorn\",\n",
    "        \"fastapi\",\n",
    "        \"pydantic\",\n",
    "        \"typing\",\n",
    "        \"pydantic\",\n",
    "        \"langchain_openai\",\n",
    "        \"langchain_core\",\n",
    "        \"langchain_community\",\n",
    "        \"langchain_chroma\",\n",
    "    ],\n",
    "    verbose=False,\n",
    "    upgrade=False,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "7f9065ea",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Environment variables have been set successfully.\n"
     ]
    }
   ],
   "source": [
    "# Set environment variables\n",
    "from langchain_opentutorial import set_env\n",
    "\n",
    "set_env(\n",
    "    {\n",
    "        \"OPENAI_API_KEY\": \"\",\n",
    "        \"LANGCHAIN_API_KEY\": \"\",\n",
    "        \"LANGCHAIN_TRACING_V2\": \"true\",\n",
    "        \"LANGCHAIN_ENDPOINT\": \"https://api.smith.langchain.com\",\n",
    "        \"LANGCHAIN_PROJECT\": \"FastAPI-Serving\",\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "690a9ae0",
   "metadata": {},
   "source": [
    "You can alternatively set API keys such as ```OPENAI_API_KEY``` in a ```.env``` file and load them.\n",
    "\n",
    "[Note] This is not necessary if you've already set the required API keys in previous steps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "4f99b5b6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load API keys from .env file\n",
    "from dotenv import load_dotenv\n",
    "\n",
    "load_dotenv(override=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "250917b3",
   "metadata": {},
   "source": [
    "## What is FastAPI\n",
    "FastAPI is a modern, high-performance web framework for building APIs with Python, based on standard Python type hints.\n",
    "\n",
    "Key features include:\n",
    "\n",
    "- Speed: Built on Starlette and Pydantic, it is fully compatible with these tools and delivers extremely high performance—on par with NodeJS and Go—making it one of the fastest Python frameworks available.\n",
    "- Fast coding: Increases feature development speed by approximately 200% to 300%.\n",
    "- Fewer bugs: Reduces human (developer) errors by around 40%.\n",
    "- Intuitive: Offers excellent editor support with autocomplete everywhere, reducing debugging time.\n",
    "- Easy: Designed to be simple to use and learn, cutting down on the time needed to read documentation.\n",
    "- Robust: Provides production-ready code along with automatically generated interactive documentation.\n",
    "- Standards-based: Built on open, fully compatible standards for APIs, such as OpenAPI (formerly known as Swagger) and JSON Schema."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73317422",
   "metadata": {},
   "source": [
    "### FastAPI Features\n",
    "Key Features:\n",
    "\n",
    "- Supports asynchronous programming.\n",
    "- Provides automatically updating interactive API documentation (Swagger UI), allowing you to interact with your API directly.\n",
    "- Boosts coding speed with excellent editor support through autocomplete and type checking.\n",
    "- Seamlessly integrates security and authentication, enabling use without compromising your database or data models while incorporating numerous security features—including those from Starlette.\n",
    "- Automatically handles dependency injection, making it easy to use.\n",
    "- Built on Starlette and Pydantic, ensuring full compatibility."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26ebc4ad",
   "metadata": {},
   "source": [
    "### How to run a server\n",
    "\n",
    "You can find the API documentation in the ```/docs``` path and interact with it directly via the ```Try it out``` button.\n",
    "\n",
    "To spin up a live server, you can copy the code to a ```.py``` file and run it by typing ```uvicorn [file name]:[FastAPI instance] --reload``` in a shell.\n",
    "\n",
    "For this tutorial, we'll temporarily run the server from the ```.ipynb``` file with the following code\n",
    "```python\n",
    "import uvicorn\n",
    "import nest_asynci\n",
    "\n",
    "nest_asyncio.apply()\n",
    "uvicorn.run(app)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88e43ab8",
   "metadata": {},
   "source": [
    "## FastAPI Fast Tutorial\n",
    "Quickly learn how to communicate with the API via FastAPI.\n",
    "- Create an instance of FastAPI with ```FastAPI()```.\n",
    "- Define a path operation decorator to communicate with the path by setting the HTTP Method on the path.\n",
    "\n",
    "### How to run code\n",
    "When you run the code block, it's loading infinitely, which means the server is running.\n",
    "\n",
    "We recommend testing the API at ```http://127.0.0.1:8000/docs```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e116ce80",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:     Started server process [26086]\n",
      "INFO:     Waiting for application startup.\n",
      "INFO:     Application startup complete.\n",
      "INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:     127.0.0.1:54044 - \"GET / HTTP/1.1\" 200 OK\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:     Shutting down\n",
      "INFO:     Waiting for application shutdown.\n",
      "INFO:     Application shutdown complete.\n",
      "INFO:     Finished server process [26086]\n"
     ]
    }
   ],
   "source": [
    "import uvicorn\n",
    "import nest_asyncio\n",
    "from fastapi import FastAPI\n",
    "\n",
    "app = FastAPI()  ## create FastAPI instance\n",
    "\n",
    "\n",
    "# FastAPI decorators are used to set routing paths\n",
    "@app.get(\"/\")\n",
    "def read_root():\n",
    "    return \"hello\"\n",
    "\n",
    "\n",
    "nest_asyncio.apply()\n",
    "uvicorn.run(app)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df4b49e6",
   "metadata": {},
   "source": [
    "### Define Path Parameters\n",
    "\n",
    "- You can set parameters on a path and use them as variables inside a function by setting the arguments of the function.\n",
    "- You can declare the type of the path parameters in your function using Python's standard type annotations.\n",
    "- FastAPI will automatically ‘parse’ the request to validate the type of the data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19295c2f",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "503b5ab9",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:     Started server process [26086]\n",
      "INFO:     Waiting for application startup.\n",
      "INFO:     Application startup complete.\n",
      "INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:     127.0.0.1:54093 - \"GET / HTTP/1.1\" 404 Not Found\n",
      "INFO:     127.0.0.1:54094 - \"GET /chat/123 HTTP/1.1\" 200 OK\n",
      "INFO:     127.0.0.1:54094 - \"GET /chat/hello HTTP/1.1\" 422 Unprocessable Entity\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:     Shutting down\n",
      "INFO:     Waiting for application shutdown.\n",
      "INFO:     Application shutdown complete.\n",
      "INFO:     Finished server process [26086]\n"
     ]
    }
   ],
   "source": [
    "app = FastAPI()  # create FastAPI instance\n",
    "\n",
    "\n",
    "# Declare route parameters by adding parameters to the route.\n",
    "@app.get(\"/chat/{chat_id}\")\n",
    "def read_chat(chat_id: int):  # Pass the path parameter as a parameter of the function.\n",
    "    return {\"chat_id\": chat_id}\n",
    "\n",
    "\n",
    "nest_asyncio.apply()\n",
    "uvicorn.run(app)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "241cd7dd",
   "metadata": {},
   "source": [
    "### Define Query Parameters\n",
    "- If you declare a function parameter other than as part of a path parameter, FastAPI automatically interprets it as a query parameter.\n",
    "- Query parameters can be declared as optional parameters by setting their default value to ```None```.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "d692367e",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:     Started server process [26086]\n",
      "INFO:     Waiting for application startup.\n",
      "INFO:     Application startup complete.\n",
      "INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)\n",
      "INFO:     Shutting down\n",
      "INFO:     Waiting for application shutdown.\n",
      "INFO:     Application shutdown complete.\n",
      "INFO:     Finished server process [26086]\n"
     ]
    }
   ],
   "source": [
    "app = FastAPI()\n",
    "\n",
    "\n",
    "# Declare the path parameter and the query parameter.\n",
    "@app.get(\"/chat/{chat_id}\")\n",
    "def read_item(chat_id: int, item_id: int, q: str | None = None):\n",
    "    # item_id, q is the query parameter, and q is an optional parameter.\n",
    "    return {\"chat_id\": chat_id, \"item_id\": item_id, \"q\": q}\n",
    "\n",
    "\n",
    "nest_asyncio.apply()\n",
    "uvicorn.run(app)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e6f8bfc",
   "metadata": {},
   "source": [
    "### Define Request Model\n",
    "- It can be defined using the ```Pydantic``` model.\n",
    "- Request is the data sent from the client to the API. Response is the data that the API sends back to the client.\n",
    "- You can declare the request body, path, and query parameters together.\n",
    "\n",
    "**Note:** It is not recommended to include a body in a ```GET``` request."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "id": "c2bea114",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:     Started server process [26086]\n",
      "INFO:     Waiting for application startup.\n",
      "INFO:     Application startup complete.\n",
      "INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)\n",
      "INFO:     Shutting down\n",
      "INFO:     Waiting for application shutdown.\n",
      "INFO:     Application shutdown complete.\n",
      "INFO:     Finished server process [26086]\n"
     ]
    }
   ],
   "source": [
    "from pydantic import BaseModel\n",
    "\n",
    "app = FastAPI()\n",
    "\n",
    "\n",
    "# Define an Item class that is the Request Model.\n",
    "class Item(BaseModel):\n",
    "    name: str\n",
    "    description: str | None = None  # Optionally set it by declaring a default value.\n",
    "    price: float\n",
    "    tax: float | None = None\n",
    "\n",
    "\n",
    "@app.post(\"/items/{item_id}\")\n",
    "async def create_item(item_id: int, item: Item, q: str | None = None):\n",
    "    result = {\"item_id\": item_id, **item.model_dump()}\n",
    "    # if q exists, add q to result\n",
    "    if q:\n",
    "        result.update({\"q\": q})\n",
    "    # add price_with_tax if tax exists\n",
    "    if item.tax is not None:\n",
    "        price_with_tax = item.price + item.tax\n",
    "        result.update({\"price_with_tax\": price_with_tax})\n",
    "    return result\n",
    "\n",
    "\n",
    "nest_asyncio.apply()\n",
    "uvicorn.run(app)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78b8164f",
   "metadata": {},
   "source": [
    "### Define Response Model\n",
    "\n",
    "You can define the return type by adding the ```response_model``` parameter to the path operation decorator.\n",
    "\n",
    "This allows you to exclude sensitive data received from the input model from the output.\n",
    "\n",
    "FastAPI provides the following features when setting the output model\n",
    "- Converting output data to type declarations\n",
    "- Data validation\n",
    "- Add JSON schema to the response in the Swagger UI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "id": "657c92d7",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:     Started server process [26086]\n",
      "INFO:     Waiting for application startup.\n",
      "INFO:     Application startup complete.\n",
      "INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)\n",
      "INFO:     Shutting down\n",
      "INFO:     Waiting for application shutdown.\n",
      "INFO:     Application shutdown complete.\n",
      "INFO:     Finished server process [26086]\n"
     ]
    }
   ],
   "source": [
    "from typing import Any\n",
    "\n",
    "app = FastAPI()\n",
    "\n",
    "\n",
    "class PostIn(BaseModel):\n",
    "    postId: str\n",
    "    password: str\n",
    "    description: str | None = None  # Optionally set it by declaring a default value.\n",
    "    content: str\n",
    "\n",
    "\n",
    "class PostOut(BaseModel):\n",
    "    postId: str\n",
    "    description: str | None = None  # Optionally set it by declaring a default value.\n",
    "    content: str\n",
    "\n",
    "\n",
    "@app.post(\"/posts\", response_model=PostOut)\n",
    "async def create_Post(post: PostIn) -> Any:\n",
    "    return post\n",
    "\n",
    "\n",
    "nest_asyncio.apply()\n",
    "uvicorn.run(app)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95e17865",
   "metadata": {},
   "source": [
    "## FastAPI Serving of LangChain\n",
    "- Try serving a langchain with the fastAPI.\n",
    "- Use what you have learnt above.\n",
    "- Implement stream output in the fastAPI."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "28944b6b",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:     Started server process [26086]\n",
      "INFO:     Waiting for application startup.\n",
      "INFO:     Application startup complete.\n",
      "INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:     127.0.0.1:56950 - \"POST /add-contents HTTP/1.1\" 200 OK\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:     Shutting down\n",
      "INFO:     Waiting for application shutdown.\n",
      "INFO:     Application shutdown complete.\n",
      "INFO:     Finished server process [26086]\n"
     ]
    }
   ],
   "source": [
    "from typing import List\n",
    "from fastapi import FastAPI\n",
    "from dotenv import load_dotenv\n",
    "from langchain_chroma import Chroma\n",
    "from pydantic import BaseModel, Field\n",
    "from langchain_openai import ChatOpenAI, OpenAIEmbeddings\n",
    "from fastapi.responses import StreamingResponse\n",
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "from langchain_core.runnables import RunnablePassthrough\n",
    "from langchain_core.output_parsers import JsonOutputParser\n",
    "\n",
    "DB_PATH = \"../data/chroma_db\"\n",
    "\n",
    "load_dotenv()\n",
    "\n",
    "\n",
    "# Define the chat output data structure.\n",
    "class ChatReturnType(BaseModel):\n",
    "    question: str = Field(description=\"question\")\n",
    "    answer: str = Field(description=\"answer\")\n",
    "\n",
    "\n",
    "# Defines the chat stream output data structure.\n",
    "class ChatReturnStreamType(BaseModel):\n",
    "    question: str = Field(description=\"question\")\n",
    "    answer: str = Field(description=\"answer\")\n",
    "\n",
    "\n",
    "# Define the Add contents input data type.\n",
    "class AddContentsInType(BaseModel):\n",
    "    content: List[str]\n",
    "    source: List[dict]\n",
    "\n",
    "\n",
    "# Define the Add contents output data type.\n",
    "class AddContentsOutType(BaseModel):\n",
    "    content: List[str]\n",
    "    source: List[dict]\n",
    "    id: List[str]\n",
    "\n",
    "\n",
    "chroma = Chroma(\n",
    "    collection_name=\"FastApiServing\",\n",
    "    persist_directory=DB_PATH,\n",
    "    embedding_function=OpenAIEmbeddings(),\n",
    ")\n",
    "\n",
    "retriever = chroma.as_retriever(\n",
    "    search_kwargs={\n",
    "        \"k\": 4,\n",
    "    }\n",
    ")\n",
    "\n",
    "parser = JsonOutputParser(pydantic_object=ChatReturnType)\n",
    "\n",
    "prompt = ChatPromptTemplate(\n",
    "    [\n",
    "        (\"system\", \"You are a friendly AI assistant. Answer questions concisely.’\"),\n",
    "        (\n",
    "            \"system\",\n",
    "            \"Answer the question based only on the following context: {context}\",\n",
    "        ),\n",
    "        (\"user\", \"#Format: {format_instructions}\\n\\n#Question: {question}\"),\n",
    "    ]\n",
    ")\n",
    "prompt = prompt.partial(format_instructions=parser.get_format_instructions())\n",
    "\n",
    "llm = ChatOpenAI(model=\"gpt-4o-mini\", temperature=0)\n",
    "\n",
    "chain = (\n",
    "    {\"context\": retriever, \"question\": RunnablePassthrough()}\n",
    "    | prompt\n",
    "    | llm\n",
    "    | JsonOutputParser()\n",
    ")\n",
    "\n",
    "app = FastAPI()\n",
    "\n",
    "\n",
    "@app.post(\"/invoke\", response_model=ChatReturnType)\n",
    "def sync_chat(message: str):\n",
    "    response = chain.invoke(message)\n",
    "    return response\n",
    "\n",
    "\n",
    "@app.post(\"/ainvoke\", response_model=ChatReturnType)\n",
    "async def async_chat(message: str):\n",
    "    response = await chain.ainvoke(message)\n",
    "    return response\n",
    "\n",
    "\n",
    "@app.post(\"/stream\", response_model=ChatReturnStreamType)\n",
    "def sync_stream_chat(message: str):\n",
    "    def event_stream():\n",
    "        try:\n",
    "            for chunk in chain.stream(message):\n",
    "                if len(chunk) > 0:\n",
    "                    yield f\"{chunk}\"\n",
    "        except Exception as e:\n",
    "            yield f\"data: {str(e)}\\n\\n\"\n",
    "\n",
    "    return StreamingResponse(event_stream(), media_type=\"text/event-stream\")\n",
    "\n",
    "\n",
    "@app.post(\"/astream\", response_model=ChatReturnStreamType)\n",
    "async def async_stream_chat(message: str):\n",
    "    async def event_stream():\n",
    "        try:\n",
    "            async for chunk in chain.astream(message):\n",
    "                if len(chunk) > 0:\n",
    "                    yield f\"{chunk}\"\n",
    "        except Exception as e:\n",
    "            yield f\"data: {str(e)}\\n\\n\"\n",
    "\n",
    "    return StreamingResponse(event_stream(), media_type=\"text/event-stream\")\n",
    "\n",
    "\n",
    "@app.post(\"/add-contents\", response_model=AddContentsOutType)\n",
    "async def add_content(input: AddContentsInType):\n",
    "    id = chroma.add_texts(input.content, metadatas=input.source)\n",
    "    output = input.model_copy(update={\"id\": id})\n",
    "    return output\n",
    "\n",
    "\n",
    "@app.post(\"/async-add-contents\", response_model=AddContentsOutType)\n",
    "async def async_add_content(input: AddContentsInType):\n",
    "    id = await chroma.aadd_texts(input.content, metadatas=input.source)\n",
    "    output = input.model_copy(update={\"id\": id})\n",
    "    return output\n",
    "\n",
    "\n",
    "nest_asyncio.apply()\n",
    "uvicorn.run(app)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "langchain-opentutorial-k6AU65mE-py3.11",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
